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Abstract 

One of the main issues in the design of sensor networks is energy efficient communication of time- 
critical data. Energy wastage can be caused by failed packet transmission attempts at each node due to 
channel dynamics and interference. Therefore transmission control techniques that are unaware of the 
channel dynamics can lead to suboptimal channel use patterns. In this paper we propose a transmission 
controller that utilizes different "grades" of channel side information to schedule packet transmissions 
in an optimal way, while meeting a deadline constraint for all packets waiting in the transmission queue. 
The wireless channel is modeled as a finite-state Markov channel. We are specifically interested in the 
case where the transmitter has low-grade channel side information that can be obtained based solely 
on the ACK/NAK sequence for the previous transmissions. Our scheduler is readily implementable and 
it is based on the dynamic programming solution to the finite-horizon transmission control problem. 
We also calculate the information theoretic capacity of the finite state Markov channel with feedback 
containing different grades of channel side information including that, obtained through the ACK/NAK 
sequence. We illustrate that our scheduler achieves a given throughput at a power level that is fairly 
close to the fundamental limit achievable over the channel. 

Keywords: Sensor networks, Markov channel, automatic repeat request (ARQ), transmission scheduler. 

I. Introduction 

Energy efficient communication is one of the key concerns in the design of wireless sensor 
networks. Without any effort for adapting to the variability in the channel, the system resources 
are consumed inefficiently. For instance, if bad channel conditions are not anticipated, a high 
fraction of the node energy can be consumed by multiple retransmissions per correctly decoded 
packet. To avoid such inefficiencies, sensor nodes use transmission schedulers. The objective of 
an energy-efficient transmission scheduler is to reliably communicate data using minimal amount 
of energy, while meeting deadline and/or throughput constraints. Design of efficient transmission 
schedulers is challenging - especially in sensor networks - due to limited computational resources 
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and the lack of perfect channel side information (CSI) at the transmitter. Transmitters used in 
sensor networks often do not have access to advanced modulation techniques, which rules out 
the use of sophisticated adaptive transmission schemes. 

Consequently we focus our attention on transmission schemes that do not require a high 
computational power. We propose a transmission scheduler that utilizes different "grades" of 
CSI on the state of a FSMC to schedule packet transmissions in order to meet a deadline 
constraint for the packets waiting in the transmission queue. We are specifically interested in 
the highly imperfect CSI that can be obtained based solely on the ACK/NAK sequence for the 
past transmissions. Note that, this level of information is available at the link layer in almost 
all wireless networks without any extra effort (such as sending special non-information carrying 
physical-layer pilot symbols over the channel). Our transmitter has a single transmit power 
level and the coding and modulation schemes are fixed. In every transmission opportunity, the 
transmitter can choose to attempt the packet transmission or defer it. 

Our scheduler is based on the dynamic programming (DP) solution to a finite-horizon transmis- 
sion control problem. We show how a simple version of it can be implemented in sensor nodes. 
We also calculate the capacity-cost function of the FSMC with feedback containing different 
grades of CSI. To our best knowledge, the capacity of channels with a feedback that is a random 
function of the channel state (e.g., ACK/NAK) has not been addressed before. We finally evaluate 
the performance of our scheduler and illustrate that it achieves a given throughput at a power 
level that is fairly close to the fundamental limit achievable over the channel. 

In the transmission scheduling problem, the parameter we minimize is the number of trans- 
mission attempts, subject to a deadline constraint for the packets in the queue. One can realize 
that, with the limitation of binary power control (transmit or do not transmit), the number of 
transmissions is proportional to the energy consumed per correctly decoded packet and the 
deadline constraint can be translated into a throughput constraint at every point in timqlj. Hence 
the problem can be viewed as minimizing the energy per packet subject to a throughput constraint. 

Transmission strategies based on channel estimation in FSMCs have been considered by Zorzi 
and Rao [HI as well as Chiasserini and Meo [|2l. These papers assume a two-state Markov chain 
and detect the bad channel state upon receipt of a NAK. Both schemes reduce the transmission 
rate and constantly probe the channel as a response. In [|2l, the transmitter switches to a greedy 

'For instance if tlie buffer contains 5 packets witli a deadline constraint of 1 sec, tlien the scheduler has to guarantee a 
minimum of 5 packets/sec for the next second 
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transmission mode when the buffer level exceeds a certain threshold, regardless of the channel 
state. Our scheme is based on a dynamic program, which takes the queue and the estimated 
channel states jointly into account to schedule packet transmissions in an optimal manner. 
Johnston and Krishnamurthy [3J give an algorithm that minimizes the transmission energy and 
latency while transmitting over a fading channel by formulating the problem as a partially 
observed Markov decision process (POMDP) search problem. However, their threshold-based 
policy result is optimal for a 2-state channel only. In addition, the channel model is assumed to 
have a unity packet loss probability in one state. This may not seem like a fundamental difference, 
but in such a scenario, an ACK implies a "good" state. The corresponding time dependence is 
finite and the associated solution can exploit it. Our results are valid for a general FSMC. 

Uysal, et. al., flU and Zafer, et. al., O have considered rate control policies for transmission 
scheduling similar to ours. However, ||4l uses a static channel model for deriving the control 
policy. On the other hand, [5| considers a Markov channel, but assumes knowledge of the channel 
state prior to transmission. In our model, this information is not available to the controller. 
Haleem, et. al., [6] use ACK/NAK feedback in a learning automata algorithm to schedule 
transmissions. This method is shown to converge to the optimal throughput in stationary channels. 
Ho, et. al., Q give a sub-optimal rate adaptation scheme to maximize throughput that uses 
ACK/NAK feedback. Karmokar, et. al., |[8l pose the problem of rate adaptation in Type-I Hybrid 
ARQ systems as a POMDP and provide some heuristic solutions. The authors develop this idea 
further and propose a linear programming approach to solve the POMDP problem IfTOl . 

In the second part of the paper, we study fundamental limitations of FSMCs with feedback 
containing perfect and imperfect CSI. We will use tools from information theory to derive 
the capacity results for power constrained channels. Capacity of channels with CSI is a well 
studied topic. Goldsmith and Varaiya ifTTI gave the Shannon capacity of such channels with 
instantaneous CSI at the transmitter and receiver. In this case, CSI implies accurate knowledge 
of the instantaneous state of the Markov channel. The optimal power adaptation for such a 
channel model implies "water-filling" in time, which is analogous to the water-filling result in 
frequency in a frequency- selective channel [fT2ll . Viswanathan [fT3l extended these results for 
delayed perfect CSI at the transmitter. Yiiksel and Tatikonda derived the capacity of a FSMC 
with imperfect deterministic feedback lfT4]| . Imperfect deterministic feedback implies that the 
transmitter has inaccurate (quantized) information about the channel state. The quantizer used is 
a deterministic function of the channel state. Although these results are useful for a wide variety 
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of capacity calculations, they can not be applied directly to channels with discrete power levels 
and a long term average power constraint on the input symbols. This is due to the fact that the 
transmitter might have to defer transmission in order to meet the power constraints. 

To incorporate skipped transmission attempts in the classical capacity results, we introduce a 
new discrete channel, which we call channel with vacation. We associate a channel with vacation 
to each state of the FSMC, and calculate the capacity-cost function for this special Markov 
channel. The general problem of finding the capacity of a feedback channel with memory has 
been notoriously difficult in information theory; and the problem of feedback capacity containing 
a CSI, which is a random function of the channel state has not been addressed before. We use 
this kind of a feedback in our channel model because the probability of a transmission success 
is channel state dependent, which implies that the feedback is a random function of the channel 
state. We generalize [[T4ll and combine it with [[T3l and IfTSl in order to calculate the capacity- 
cost function of our channel with feedback. We compare this capacity-cost function with the 
performance achieved by our scheduler and observe that for a given throughput requirement, the 
power expended by our scheme is close to the theoretical limit of the system. 

The paper is organized as follows. We give our system model and the assumptions in Section|Ill 
We present the DP formulation of the problem in Section |III1 We derive the theoretical bounds 
on the system performance in Section |IVl In Section |Vl we analyze the performance of the DP 
scheduler and compare it with the theoretical bounds. We wrap up with a summary in Section IVTl 

IL System Model 

The system model used in this work is shown in Figure [B Here, we consider a single point-to- 
point wireless link with a feedback path. The transmitter has B packets in a queue to be sent to the 
receiver within T time slots. The transmitter is assumed to transmit at a fixed rate and we define 
a time slot as the time it takes to transmit a packet and receive the associated ACK/NAK. The 
controller, which is attached to the transmitter, determines the decision to attempt a transmission 
or to defer transmission of a packet in every time slot. For the A;th time slot, we denote this 
packet (or lack of it) as Xk- The receiver decodes the packet Uk and sends feedback Ck about the 
channel state to the controller over an error-free channel. The different types of feedback will be 
discussed shortly. At time k, the controller has access to the queue state information (i.e., number 
of packets in the queue, g^) and the feedback c^. We use the following notation for a sequence, 
Cm — {cm, Cm+i, • • • , c„}. The information vector = {Qq^Cq} is defined as the information 
available at time k to the controller. The controller makes control decision Uk G {0, 1} based 
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on the information vector, J^. Here '0' corresponds to the decision not to transmit and '1' 
corresponds to the decision to transmit packet Xk- 

The channel model that we consider for this problem is a FSMC. The channel state at time 
k is denoted by Sk E S, determines the packet loss probability during the A;th time slot. The 
state space of the Markov channel, 5 is a discrete set containing the different channel states 
of the FSMC. Each channel state s E S has a certain packet loss probability denoted by e^. 
The channel state is assumed to be constant during the period of packet transmission, i.e., one 
time slot. The steady state probability of state s is represented by tt(s). While not as simple 
as a Bernoulli or an independent loss channel model, Markov models are used to approximate 
channels with memory. FSMCs were first used to model a channel with bursty errors |[T6ll . and 
are popular in the literature IfTTll . We assume that the controller has accurate estimates of the 
channel parameters, i.e., transition probabilities and the packet loss probabilities associated with 
each state. Estimation of these channel parameters is beyond the scope of this paper. We note 
that this topic has been treated widely in literature, where an FSMC with unknown parameters 
is posed as a hidden Markov model (HMM). Iterative procedures for the Maximum Likelihood 
estimation of HMM parameters are well-understood, e.g., Baum- Welch algorithm IfTSl . 

The receiver is assumed to have an error detection scheme with a negligible probability of 
undetected error. The decoder identifies whether a packet is decoded correctly or not, i.e., yk = Xk 
or Uk 7^ Xk- Packet error process at time k is denoted by Zk, where = I for a correct 
transmission and Zk = for an incorrect transmission. The receiver sends the feedback containing 
the CSI to the controller on a channel assumed to be error-free. The three cases/grades of receiver 
feedback to the controller considered in this paper are: 

1) Non-causal Perfect CSI: the controller knows the instantaneous channel state when it 
determines Uk, i.e., = Sk and 4 = {q^, s^}. 

2) Causal Perfect CSI: the controller knows the delayed channel state when it determines 
Uk, i.e., Ck = Sfe_i and h = {qq, Sq"^}- 

3) Causal Partial CSI: in this case, the controller knows the ACK/NAK for the previous 
transmissions, i.e., Ck = Zk-i and Ik = {qq^Zq'^}. 

For clarity of presentation, we use a unit delay for the causal CSI cases. The results contained 
in this paper can be easily extended to a general fixed delay. In the following section, we construct 
the finite-horizon DP solution. In Section |IVl we keep the basic structure of this system model 
to derive the theoretical bounds. 
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III. The Dynamic Programming Solution 
In this section we develop an algorithm/decision rule for the controller that minimizes the 

energy expended by the transmitter and the receiver for the successful delivery of a certain 

number of data packets while maintaining a deadline requirement. Intuitively, without a deadline 

constraint, the controller would be inclined to transmit at times only when it is almost sure of the 

good channel state to achieve energy efficiency. However, with the deadline constraint, if there 

are packets remaining in the queue close to the deadline, it may need to take chances with the 

bad state occasionally as well. Thus the controller has to make decisions on packet transmission, 

based jointly on the queue and channel states. Note that the controller needs to consider the 

effect of the current decision on future decisions. For instance, if the controller decides not to 

transmit at a time slot, there will be no feedback on the channel state for that time slot. This 

makes the available information more outdated, affecting the success of subsequent decisions 

made. In our system the CSI used by the controller is provided solely by the receiver. We assume 

that the initial queue state is also known by the receiver, hence the receiver also has the entire 

information vector. Consequently, the receiver has the knowledge of whether a transmission will 

be attempted at any given point in time and it can remain inactive to conserve energy during 

skipped transmission attempts. 

Since the packet-loss process is a stochastic process, this problem can be viewed as one of 

sequential decision making under uncertainty. In our system model, both the queue evolution and 

channel evolution have a Markov structure. The queue state can be observed completely by the 

controller, however for the channel state, we have different cases of observation (complete and 

partial). The combination of these factors necessitates the use of a finite horizon DP approach |fT9l 

to achieve energy optimality and meet the deadline constraint at the same time. 
A. The Dynamic Program 

Using the notation introduced in the previous section, we can write the state equation for the 

queue occupancy qk at the beginning of time slot /c = 0, 1, . . . , T — 1, as follows, 

qk+i = f{qk, Sk, Uk) = qk - ^es^Uk, (1) 

where Uk G {0, 1}. The packet loss process is denoted by 1^^^, where le^^ = with probability 
e., and 1. =1 otherwise. 

Our objective is to minimize the energy consumption of the transmitter while transmitting B 
packets over T time slots. Consequently, the cost function should be proportional to the total 
energy of the transmissions required to transmit all packets correctly. In order to do this, we set 
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the cost incurred at time k, denoted by gk^Qk, Sk,Uk), equal to Uk- When the controller decides 
to transmit, it incurs a cost of 1 unit. On the other hand, if the controller decides to defer 
transmission, then it does not incur any cost. Since the packet loss is a stochastic process, there 
is always a non-zero probability associated with not transmitting all packets correctly by the 
deadline. To make this undesirable, we use a terminal cost grigr) = Cqt, C ^ 0. Terminal 
cost implies that if there are Qt packets left in the buffer at the end of time T, then the controller 
will incur a cost of Cqt- One would expect that the decision rule and hence the performance is 
strongly tied to the choice of C, but we show that, somewhat surprisingly, in most simulations 
the choice of C has little effect on the performance of the scheduler as long as C > 10. We 
can express the expected cost incurred from the (T — l)th stage to termination, also called the 
cost-to-go function Jt~i{It~i) as. 



Jt^i{It~i) = rain I E [^^(/(gT-i, st-i, mt-i)) + fi'T-i(gT-i, st-i, MT-i)|/r-i, ^t-i] 



The cost-to-go Jk{h) for stages k = 0, 1,...,T — 2, can be expressed iteratively by using 
successive cost-to-go functions Jk+i{Ik+i), 

Jfc(4) = min <^ E [Jk+iih, Qk+i, Ck+i,Uk) + gk{qk, Sk,Uk)\Ik,Uk]> 



We note that these optimization problems reduce to a DP with perfect state information when 
non-causal perfect CSI is available. In case of causal CSI (perfect and partial) these problems 
are treated as DP with imperfect state information. At each k = 0,1, . . . ,T — 1, the optimal 
policy fil{Ik) rnaps the information vector Ik to the control action Uk E {0, 1} that minimizes 
the cost-to-go given in Q and ([3]). The optimal policy is obtained recursively, we first solve the 
optimization problem Q for all possible values of the information vector It-i to get /i^„]^(/r-i). 
The corresponding value of J^_i{It^i) is used to calculate yU^_2(/r-2) in © and J^_2(/t-2)- 
This process is continued till Jq (/o) = Joilo, cq) is found. We note that, in general, at each stage 
k the state space size of Ik will grow exponentially. However, if the components of Ik follow a 
Markov transition (e.g., perfect CSI feedback) then only the most recent state observations are 
required for determining the control decision keeping the state space size constant. 
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Next, we derive the DP equations for the case when causal partial CSI is available to the 
controller. The DP equations for causal and non-causal perfect CSI are simple applications 
of ^ and ([3]) and are omitted due to space constraints. 
B. DP Equation for Causal Partial CSI at the Controller 

Recall that with causal partial CSI, Ik = {(li,z^~^}- The queue state, g^, follows a Markov 
transition, so we can reduce the information vector to Ik = {(lk^z\~^}. We cannot apply the 
same reduction to the ACK/NAK sequence, Zk. This is due to the fact that, even though Zk is a 
function of the channel state s^, in general, the probability distribution of Zk does not depend 
only on Zk^i but all the past observations, z\~^ . The problem with directly applying the DP 
approach, Q and ([3]), in this case is that the state space of the information vector will expand 
exponentially with the received observations. To avoid this problem, we introduce a new quantity 
Wk = {wk{l), . . . , which is the conditional state distribution given the past sequence, 

i.e., iVk{s) = p (sk = s I -Zi^^). The conditional state distribution follows a Markov transition, 
Wk = ^{wk-i, Zk-i, Uk-i), which can be derived using a straightforward application of Bayes' 
rule. Since this quantity depends on observations and control actions from the previous stage 
only, the controller needs to track only the most recent value of this variable. Evaluating the 
expectation in Q and dS]), using w^, we write the cost-to-go function Jk. For the terminal stage, 

Jr-i(gr-i,it>T-i) = min |mt-i + ^wr-i(s) (e^Cgr-i + (1 - e^)C(gr-i - }, (4) 

s£S 

and for the intermediate stages, k = 0, . . . ,T — 2, 

Jk{qk, Wk) = min i + Wk{s)esJk+i{qk, ^{wk, 0, Uk)) 

+ ^Wk{s){l -es)Jk+i{qk -Ufc,<l>(iUfe,l,nfc))|. (5) 

Note that (01) and © are valid for any discrete set S. The recursive relation for is given by, 

^/ N w;fc-iA(zfc-i,«fc-i)P 

Wk = 9{wk-i, Zk-1, Uk-i) = — — ^7, (6) 

Wk-iA{zk-i,Uk-i)l 

with Wo initialized as the steady state distribution of states, {vr^jVs G S}. Here A{zk-i,Uk-i) 
is a diagonal \S\ x \S\ matrix with ith diagonal term defined as, 



p{zk-i I Sk-i = i,Uk^i, 



f 

e,, Zk-\ = 0,Mfc-i = 1 

1 — ^i, Zk-i = l,Uk-l = 1 

1, Uk-i = 
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P is the transition probability matrix for the FSMC, and 1 = {1,...,!} is a |iS| -dimensional 

vector. We note here that when u^-i = 0, there is no transmission in the {k — l)th slot and 

Zk-i is not known. When this "gap" appears in the ACK/NAK sequence, Wk is updated by 

vector multiplication with the transition matrix P. Intuitively, the conditional state estimate will 

be less accurate when there are gaps in the ACK/NAK sequence which leads to a degradation in 

the performance of the DP- scheduler. As expected, with increasing gap size, the state estimate 

converges to the steady state probability. 

C. Implementation Issues 

In this section, we discuss some implementation issues associated with the DP-scheduler. 

The computation required to solve the DP equations is beyond the computational capability of 

most sensor nodes. To bypass this problem, we propose the use of lookup tables to implement 

the scheduler. This approach requires memory resources rather than computation resources. A 

lookup table TiY(P, ei, 62, . . . , e|5|, /c, g^, lUfc) which has been pre-computed and loaded in the 

sensor memory is used to determine the control rule. The table contains the control action 

for different channel and queue conditions. The controller updates according to Eq. (6) and 

looks up the control action, n^, corresponding to this vector. Since it is not practical to have such 

a table for all channel realizations, we store tables for a discrete set of channel realizations, A. 

The major design parameters on the transmitter/controller in this approach is the amount of 
memory to store the look-up table Tz^(P, ei, 62, ... , e\s\., k, g^, Wk), where {P, 61,62, ... , e|5|} G A. 
The size of memory will depend on the number of channel realizations |^| stored in the look-up 
table. In addition, each value of and k will have a set of Wk associated with it. Let each 
component of be quantized into 10 levels. If the DP algorithm provides a solution for up to 
100 data packets transmitted over a duration of up to 500 time units, the memory requirement 
will be X 50 X IQI'^I^^ Kbits (since each the control action will need only one bit of storage). 
For = 100 and \S\ = 3, the memory requirement for the controller will be 62.5 Mbytes. 

Before concluding this section, we would like to point out that there is an inherent tradeoff 
between accuracy and complexity with increasing number of states. The memory requirements 
for storing the controller action will increase exponentially with the number of FSMC states. 
On the other hand, the channel model will become more "accurate" as more channel states are 
taken into account. However, there is no universal rule for selecting the number of states in 
the FSMC model for all possible physical fading processes. We direct the reader to [il7il and 
references therein for a discussion on the selection of number of FSMC states. 
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IV. Performance Bounds 
The DP-based scheduler in Section ni provides a strategy to consume the minimum energy, 

VT, to attempt successful transmission of B packets by a specified deadline T. This corresponds 

to a throughput of B /T at power V. In this section, we relax the deadline constraint and find 

the maximum achievable asymptotic throughput subject to a power constraint V. This quantity is 

known as the capacity-cost function C{V) [|20ll . The inverse of the capacity-cost function gives 

the minimum power V for a given asymptotic throughput C llITI . linking it to the objective of 

the DP problem. Inverting the axes of V-C(V) plot will give the plot of P as a function of C. 

The capacity-cost function will give an upper bound on the achievable throughput of the 
DP-scheduler. To understand this statement, we note that the information theoretic calculations 
are asymptotic results, i.e., number of channel uses tend to infinity while meeting a throughput 
requirement. Any scheduler that transmits B packets in T time slots will achieve an asymptotic 
throughput of B/T packets/time slot by replicating itself every T time slots. However, the 
converse is not true, as ensuring an asymptotic throughput does not guarantee a throughput for 
every block of T time slots. As a result, the capacity-cost function requires less power to achieve 
the same throughput and provides an upper bound on the throughput of the DP-scheduler. 

To adapt our system for information theoretic calculations, we make some modifications to the 
system model shown in Figure |2l Since there is no deadline constraint in the problem, the queue 
in this system is assumed to be infinitely backlogged, and omitted from the figure. The channel 
state, as before, is a Markov chain {Sk E S, k = 1,2, . . .} and the CSI Ck is a function of Sk- 
The channel input is denoted hy Xk E X and output is denoted by Yk E y. The messages, chosen 
from a set of equiprobable set of messages W, are denoted by W and the decoded messages 
are denoted by W. The size of the message set W is M. Before presenting the specifics of the 
capacity calculation, we define some information theoretic quantities. 

Encoder: For a given message set W, an encoder is a sequence of code functions {fk{w, Cq~^)}a,>i, 
where the symbol to be sent at time k is given by fk{w, Cq~^). 

Channel Code: For a given message set W, the (T, M, V, e) channel code consists of: (1) 
The blocklength of the codewords is equal to T, i.e., the number of channel uses to transmit a 
message is equal to T. (2) Each codeword {fk{w, Co~^)}i<fc<T V w, Cq ~\ satisfies the constraint, 
^ YlJi=i Kfii'^^y Cg^^)) ^ where b{x) is the non-negative cost associated with the input symbol 
X E X. Therefore "P is a hard power constraint imposed on every sample outcome of the channel 
state or ACK/NAK sequence. (3) The average probability of incorrectly decoding a message is 
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bounded as ^ J2w&wP(^ w\W = w) < e. 

Capacity-Cost Function: Given < e < 1 and V > 0, a non-negative number R is an e- 
achievable rate with the average cost per symbol not exceeding V if for every 5 > there 
exists To such that if T > To, then an (T, M, "P, e) code can be found whose rate satisfies 
log M > T{R — 6). Furthermore, R is said to be achievable if it is e-achievable for all < e < 1. 
The maximum achievable rate with average cost per symbol not exceeding V is the channel 
capacity denoted by C{V), which is referred to as the capacity-cost function. 

We cannot directly apply classical channel capacity results as the system that we consider has 
special properties. The factors to consider while evaluating the channel capacity are: 

1) Skipped Channel Uses: An ON-OFF scheduler is limited to two discrete power levels 
corresponding to the decision to transmit (at a fixed power) or not to transmit. As a result, 
under a power constraint, we should consider an "outage capacity" in which the decision 
not to transmit is accounted for in the capacity calculations. 

2) Markov Channel with Delayed Feedback: Deterministic perfect and quantized feedback 

in such channels has been treated in literature |[T3l , [[T4l . but randomly quantized feedback 

has not. We have to calculate the capacity-cost function of a Markov channel with feedback 

in the form of ACK/NAK. In our channel model, ACK/NAKs are a random function of 

the channel state and hence need to be considered accordingly. 

A. Finite State Markov Erasure Channel (FSMEC) 

A FSMEC is a special case of FSMC where, during each symbol duration is one of a finite 

number of erasure channels (ECs) with Markov transitions between these ECs. The channel state 

Sk determines the channel erasure probability during the kth symbol. Note that we describe an 

entire packet (or lack of it) as a channel symbol and we model a detectable packet error using 

an erasure. In this problem, since we assume an ON-OFF transmitter, the input symbols are 

restricted to discrete power levels and have a long term average power constraint. Consequently, 

it is not possible to transmit data at every time instant. Discrete ECs fT2]| do not provide any 

provision for calculating mutual information rate under such power constraints. As a result, the 

channel model has to be designed in such a way that it incorporates the "OFF-period" of the 

transmitter in the capacity calculations. 

To integrate the idea of no transmission in our channel model, we add an input symbol 

('V') to the traditional EC as shown in Figure |3] and call the new channel erasure channel 

with vacations (ECV). Each input symbol has an associated cost (power) of 1 unit and the cost 
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required to transmit 'V is 0. The transition probabilities of 'V are chosen according to the 

statistical requirement rather than the physical signaling system. We will show in Appendix |A] 

that the capacity-cost function C(V) of the ECV in Figure [3] is VN{1 — e^). This can be 

intepreted as the capacity of an EC that is active for V fraction of time. One can observe that 

the unconstrained capacity (i.e., V = 1) of this ECV is the same as that of an EC with 2^ 

input symbols and erasure probability eg. As a result, addition of 'V to the EC does not change 

the mutual information between the input and output symbols. Therefore, we infer that 'V is a 

zero-cost, zero-information symbol which can replace the OFF-period of the transmitter for the 

capacity calculation. For the ON-period, we consider transmission of A^-bit (fixed-size) packets 

that require unit cost to transmit. Hence, there will be 2^ distinct symbols, {'1', . . . , '2^'}, 

shown in Figure [3] representing the ON-period of the transmitter. To make this model equivalent 

to the one in Section [III we make the transition and the error probabilities of the two channels 

equal. We assume that the probability of packet error is independent of the packet "content". As 

a result in state s, each packet (symbol) is transmitted correctly with probability 1 — e^. 
B. Capacity of the FSMEC 

We state the following results for the capacity-cost function of the FSMEC with causal perfect 

and partial CSI at the transmitter. Each state s E S is associated with an ECV (shown in Figure[3]) 

with parameter e^, steady state probability 7r(s) and 2^ + 1 input symbols. 

Theorem 1: The capacity-cost function for a FSMEC with transition probability matrix P, 

channel state space S, and causal perfect CSI at the encoder is given by. 



The optimization variable is V{s), which can be thought of as the power allocation policy for 
state s. It is the fraction of unit cost symbols in the codebook for state s. 
The solution to ([7]) gives the optimal power allocation policy {■p*(s), Vs E S}. Substituting this 
power allocation into the original expression gives the channel capacity. Following is a brief 
sketch of the proof, details of which can be found in Appendix |Al Each ECV, associated to a 
state in the FSMEC, has a capacity-cost function. First we find the capacity-cost function of 
the ECVs using the expression given by Verdu [15]. Next, we use the capacity expression for a 
FSMC with causal perfect CSI at the transmitter given by Viswanathan |fT3l . The capacity-cost 
function substituted in the FSMC capacity expression leads to the optimization problem ([7]). 




(7) 



s 
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We present similar results when causal partial CSI is available to the encoder. We denote 
E {0, 1}" as an ACK/NAK sequence of length n and p{z^) is the probability of = z". 
Corollary 2: The capacity-cost function for FSMEC with each state s E S associated with 
an ECV with parameter and causal partial CSI (ACK/NAK) at the encoder is given by, 

J2 Pi^i) sup {p(.„+ikr)(l - es,^^jNV{z^)} 

s.t. J2p(^i)'P(^i) < ^ & < V{z'^) < 1, Wz^ E {0, 1}". (8) 

Here, the optimization is done over V{z^), which can be thought of as the power allocation 
policy for the ACK/NAK observation z^. 

This corollary extends Theorem[T]to the case when only ACK/NAK information is available to the 
encoder. In this case the power allocation policy becomes a function of the observed ACK/NAK 
sequence instead of the last observed state. The first part of the proof, i.e., capacity-cost calcula- 
tion, remains unchanged. We change the feedback capacity expression used in Theorem [T] to one 
adapted from lfT4ll . The proof is mostly kept intact, a modification is made in the formulation of 
Fano's inequality in Eq. (fT2l) of [14j to incorporate ACK/NAK feedback at the transmitter. The 
details of the modification are given in Appendix |Bl 

The particular expression ([8]) provides an indication to a level-filling solution to the optimiza- 
tion problem. To evaluate this expression, we calculate the joint probability p(sn+i, z'^). The joint 
probability expression can be expanded as '}2s^Pi^n+i\sn)p{zn\sn) ■ ■ ■ J2si p{-^2\si)p{zi\si)p{si), 
which can be evaluated iteratively. The optimization problem is then solved over all possible se- 
quences which results in a level-filling solution for 2" states. We calculate C"(P) for different 
values of n and observe that, for most cases, C^^{V) is reasonably close to Cfsmec-arq('P)- 

Example 1: (A 2-state Markov erasure channel (2SMEC) with perfect causal CSI) We consider 
a 2SMEC with ei < £2 and transition probabilities P(l,2) = pi2 and P(2, 1) = p2i. For this 
channel, tti = — and tto = — . Applying Theorem [Tl we can find the power allocation 

^ P12+P21 ^ P12+P21 ff J b 1— » Jr- 

policy {V{1),V{2)}. The plot for {V{1),V{2)} as a function of V is shown in Figure IH For 
Case 1, when (1 - pi2)(l - Ci) + p^il - €2) > P2i{l - ^1) + (1 - P2i)(l - £2) and power 
constraint V < tti, power is allocated only to 5 = 1. This is represented by the solid blue trace. 
For V > ni, there are more opportunities to transmit then there are "good states" available. As a 
result, (V — ni) part of the power is now allocated to 5 = 2 shown by the dashed red trace. This 



Cfsmec-arq(7') = lim C^{V) = lim 

n— >oo n— >oo 
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continues till V = 1, at which point all the states are utilized for transmission. A corresponding 
trend can be shown when (1 - pi2)(l - ci) + p^il - €2) < P2i{l - ei) + (1 - P2i)(l - £2)- 
These traces can be interpreted as level-filling results for power allocation in discrete channels, 
analogous to the one in ifTTIl . 

V. Performance Evaluation 
A. Scheduler Performance and Theoretical Bounds 

We conduct simulations on FSMCs to compare the various transmission schedulers and 

compare them against their respective theoretical bounds. The metric used to compare the 

scheduler performance is termed normalized transmission cost (NTC). We define d as the 

number of data packets successfully transmitted in one time slot and NTC((i) is the number 

of transmission attempts required to achieve this. For the theoretical bounds, the capacity-cost 

function C{V), given by Theorem \T\ and Corollary |2l is plotted as a function of the power 

constraint V. Here V is equivalent to the number of packet transmission attempts made in a 

time slot. The capacity-cost function C{V) is divided by the packet length to represent the 

capacity in terms of data packets and offer a direct comparison to the NTC. 

1 ) Comparison of DP-Schedulers for FSMCs: Figures [5] and [6] show the performance com- 
parison of various DP-schedulers for 2- and 3-state Markov channels. The transition probabilities 
for the 2-state Markov channel (2SMC) are p2i = 0.1 and pi2 = 0.2. The loss probability in 
each state is ei = 0.2 and 62 = 0.8. For the 3-state Markov channel (3SMC), the transition 
probabilities are pi2 = 0.025, pn = 0.075, P21 = 0.075, P23 = 0.05, p^i = 0.05 and ^32 = 0.05. 
The loss probabilities are ei = 0.2, 62 = 0.85 and €3 = 0.95. The time horizon T for the 
simulations is 500 time slots and 10^ realizations of the channel are simulated. The packet error 
process is simulated by first generating a Markov process 500 time slots long and then setting 
the loss variable Zk according to the given state Sk. 

First, we plot the performance of DP-scheduler (solid red trace) and the corresponding bound 
(dashed red trace) for the non-causal perfect CSI case. For both cases (2SMC and 3SMC), 
the bound is tight up to the point when the deadline constraint requires a throughput of 0.2 
packets/time slot. This can be due to the fact that for a lightly loaded system, the effect of queue 
state on decision making is minimal, and the scheduler considers CSI for decision making leading 
to a performance close to the theoretic bound. In all performance curves with feedback, for a 
2SMC, a knee-point is observed when the NTC (or V) is equal to 0.33 (for 3SMC, knee points at 
NTC = 0.37 and NTC = 0.6). As the number of packets to be transmitted is increased, the cost 
will increase at a certain rate since only the good states are being used initially. However after 
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all the good states are exhausted, the scheduler will use the bad states for transmission, which 
increases cost at a faster rate as more packets will be dropped. For a throughput requirement of 
0.25 packets/time slot in a 2SMC, causal perfect CSI scheduler requires 25% more transmission 
attempts than a non-causal perfect CSI scheduler (10% for a 3SMC). 

We observe that the performance of the causal imperfect CSI scheduler or the ACK/NAK 
scheduler (solid black traces) is within 80% of the theoretical bound (dashed black traces) 
for the 2SMC (75% for the 3SMC). The bound is not very tight as the capacity calculations 
assume knowledge of the complete ACK/NAK sequence, whereas the ACK/NAK scheduler has 
access to the incomplete sequence (since ACK/NAK is unavailable for the time slots when no 
transmission is attempted). In both the cases, the ACK/NAK scheduler, at worst, requires 50% 
more transmission attempts than the non-causal perfect CSI based scheduler for achieving a 
throughput of 0.25 packets/time slot. The green trace represents a blind transmission scheme. In 
this case, the scheduler does not have any channel feedback, instead it just keeps transmitting till 
the queue is emptied out. This can be the lower bound on performance of any feedback scheme. 
The performance of this scheduler has a constant slope, since the scheduler does not adapt the 
transmission policy to the channel state. For a 2SMC, ACK/NAK scheduler outperforms the 
blind transmission scheme by up to 25% for 0.1 packets/time slot throughput (50% for a 3SMC) 
and 15% for 0.25 packets/time slot throughput (20% for a 3SMC). 

2) Look-up Table Size: Figure |7] compares the effect of look-up table size on the performance 
of the ACK/NAK scheduler. The parameters for the 2SMC are: pi2 = 0.18, p2i = 0.09, ei = 0.23 
and 62 = 0.78. The 32 MB table contains control action for |^| = 256 channel realizations and 
20 levels of w^. The parameters for the 1 MB table are \A\ = 16 and 10 levels of w^.. Both 
tables store control action for upto 100 packets transmitted over a duration of upto 500 time 
slots. We observe that the ACK/NAK schedulers using a look-up table perform close to the 
exact solution. For instance, the 32 MB table requires 7% extra transmission attempts (11.5% 
for 1 MB table) to achieve throughput of 0. 1 packets/time slot. Note that we have used uniform 
quantization for this example, however quantization methods that perform better might also exist. 

3) Time Horizon: Figure [8] compares the performance of the ACK/NAK scheduler with 
different time horizons T. The parameters for the 2SMC are: pu = 0.2, p2i = 0.1, ei = 0.1 
and 62 = 0.9. We observe that with increasing T, the performance of the ACK/NAK scheduler 
improves. A scheduler with T = 100 requires, at most, 10% more transmission attempts than a 
scheduler with T = 800. The improvement in performance can be due to the scheduler having 
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greater flexibility in scheduling transmissions with larger T. We also observe that the performance 
of the ACK/NAK schedulers (for all T) is within 75% of the theoretical bound. 

4) Terminal Cost: In Figure [9l we compare the effect of terminal cost C on the performance of 
the ACK/NAK scheduler for a 2SMC. The channel parameters are: pi2 = 0.2, p2i = 0.1, ei = 0.2 
and 62 = 0.8. For C = {10, 50, 200}, we observe that there is not much of a difference in the 
performance of the ACK/NAK schedulers formulated using these terminal costs. Other channel 
realizations also exhibit a similar behavior. This leads to the observation that the choice of C 
does not impact the performance of the scheduler, as long as C > 10. 

B. Power Penalty 

1 ) Maximum Power Penalty: We define the maximum power penalty for a feedback scheme 

• A /~iT^/xT A A ^ inT / Cost-Capacity for rate B. usine ACK/NAK feedback \ 

USmg ACK/NAK mformatlOn as A^^ = maX^lOlogio Cost-Capadty for rate R uJ, causal CSI feedback ) ■ 

The Cost-Capacity function is the inverse of the Capacity-Cost function. Maximum power penalty 
quantifies the theoretical worst-case extra power required by an ACK/NAK based scheme to 
achieve the same throughput achieved by a causal perfect CSI based scheme. We define a 
similar quantity for the blind transmission scheme. Figure [TO] compares the maximum power 
penalties incurred by ACK/NAK and blind schemes for 2SMCs as a function of channel memory 
/i = 1 — pi2 — P21- We plot this quantity for different values of loss probabilities {ei, 62}, while 
keeping the steady state probabilities {7r(l), 7r(2)} constant. The ACK/NAK based schemes do 
not incur a maximum power penalty of more than 1 dB. On the other hand, in case of no feedback 
(blind) the maximum power penalty can be as much as 4.5 dB. An interesting observation is 
that when the channel conditions are "good" (ei = 0.01, 62 = 0.1) the advantage of using an 
ACK/NAK based scheme is negligible. In this case, the difference in AdB for the two cases is 
less than 0.1 dB. This can be explained by noting that under good channel conditions, there will 

be negligible packet drops and consequently, the value of estimating the channel is minimal. 

2) Actual Power Penalty: Next, we find the actual power penalty incurred by an ACK/NAK 

scheduler, L,b ^ lOlog.o {^t"^^^^^^), for a given throughput. We define a 
similar quantity for the blind transmission scheme. Figure [TT] shows the power penalty incurred 
by the ACK/NAK based and blind schedulers for 2SMCs as a function of channel memory /i. 
We plot this quantity for different values of packet loss probabilities, {ei, 62} keeping the steady 
state probabilities {7r(l), 7r(2)} constant. For the range of channel parameters considered in our 
simulations and a throughput requirement of 0.2 packets/time slot, ACK/NAK scheduler does 
not incur a power penalty of more than 1.5 dB. On the other hand, a blind transmission scheme 
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takes a power penalty of up to 4.5 dB (for ei = and 62 = 1). 

In Figure [I2I we consider different steady state probabilities {7r(l), 7r(2)}, while keeping 

the loss probabilities constant (ei = 0.1, e2 = 0.9). The throughput requirement is set at 0.1 

packets/time slot. In the worst case, the ACK/NAK scheduler incurs a power penalty of 2.5 dB 

for TTi = 0.2 and 1x2 = 0.8 whereas the blind scheme incurs a power penalty of almost 4.5 dB. 

For fi = 0.9, in all the cases, the performance of the ACK/NAK scheduler is 2 dB better than 

the blind scheduler. Since yU is an indicator of the burstiness (as bursty channels have high values 

of /i), the improvement provided by our scheme will be more pronounced in bursty channels. 

VI. Conclusions 

We introduced a channel aware transmission scheduler to optimize energy efficiency in sensor 

networks. Our transmission scheduler meets the deadline constraint for all packets waiting in 

the transmission queue while optimizing the total energy spent on transmission. To do this, we 

used finite-horizon DP and utilized different types of CSI in the optimization process. We also 

derived the capacity-cost function of a FSMC with different types of CSI at the controller. The 

capacity-cost function can be used as a bound for the performance of the transmission schedulers. 

We showed that the performance achieved by a DP-based scheduler is close to the fundamental 

limits for a wide range of channel parameters. Indeed, for some cases, the ACK/NAK scheduler 

achieves performance that is within 80% of the theoretical bound. Comparisons between the 

performance of different grades of CSI show that the difference between performance of causal 

perfect CSI and ACK/NAK schemes for a wide range of channel parameters is not significant. 

On the other hand, blind transmission scheme in which the ACK/NAK feedback is not exploited 

to estimate the channel state incurs a significant power penalty. In addition, the maximum power 

penalty incurred by ACK/NAK feedback based schemes provide insights into the fundamental 

value of using ACK/NAK information while scheduling transmissions. For instance, in one case, 

the maximum power penalty of not using any kind of feedback is 4.5 dB. The use of ACK/NAK 

information, which is available to the transmitter in most wireless link layer protocols without 

extra effort, can reduce this power penalty to less than 1 dB. 
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Appendix 

A. Proof for TheoremUj 

The capacity-cost function of a cost constrained memoryless stationary channel is defined 

as, C(V) = sup X HX;Y). Where b\X] is the cost of transmitting symbol X. The 

E[b[X]]<V 

transmission of one bit of information requires 1/C(V) symbols at a cost of V/C{V). In order 
to find C(V), first we find the capacity per unit cost, Ccost, using the expression in ifTSl . 

Ccost = sup . , = sup — = 1 - es)N. (9) 

X E[b{X)\ X o{x) 
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In our case, b{x) = 1 for x = {'1', . . . , '2^'} and b{x) = for x = 'V. We find that for 
an ECV, D(Py|x=T | |Py|x=-v) = ... = D{Py\x='2^MPy\x='V') = 1 - e.. Using Eq. ® and 
incorporating the power constraint of V we can find the capacity-cost function of the ECV, 

C{V) = E[b{X)]a,,, = V {N{1 - e,)) . (10) 

For an average power constraint V and unit feedback delay, Cfb{V) is given by [[T3l , 

Ct-i,{V) = max I{X;Y\S,S), s.t. V 7r(5)P(S) < P, < V{s) < V, Vs. (11) 

P(X|5) ^ 

Where s E S represents the state information fed back (s = st_i) to the transmitter from the 
receiver and s E S is state at the time of transmission. V{s) is the average cost per symbol 
incurred while in state s. And we define, 

/(X; Y\S, S) = J2 ^is)I{X; Y\S,S = S) = J2 ^(s) Yl ^)^^^' Y\S = s,S = S). 

Using the capacity-cost function from Eq. (flOl) in conjunctiorj^ with the capacity expression (fTTI) . 
we have the capacity-cost function for FSMECs, 

Cfsmec(P) = sup V V 7r(S)P(5, s)(l - e,)iVP(5), s.t. V 7r(5)P(5) < P, < P(s) < P, Vs. 

5. Proof for Corollary |2] 

We calculate the capacity of an energy-constrained finite state Markov erasure channel with 

causal imperfect CSI by extending the proof given by Yiiksel and Tatikonda in [[T4|. The problem 

formulation in that work is very similar to ours. The main difference comes from the fact that 

we consider a causal probabilistic quantizer (i.e., ACK/NAK) for state feedback. In order to 

incorporate this feature, we first make some modifications to converse of the channel theorem. 

If we consider a coding scheme such that e — )■ 0, Fano's inequality gives, 

H{W\Y,'',SlZ^) < h{p,)+pJog,{M), (12) 

where pe gives the probability of error. Also, 

H{W\Y,^, Si, Zl) = H{W) - I{W- If, Si, Zl) = log2(M) - I{W; Y^ , S^ , ). (13) 

^It might appear that the capacity-cost calculations which have been made for a memoryless channel are not applicable for 
Markov chaimels (which are not memoryless). However 1131 derives capacity results which use the mutual information rate 
conditioned on the present state of the channel, which makes the channel memoryless for Markov channels. 
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Combining Eqs. (fT2l) and (fT3] ) we have, 



log2(M) ^ I{W- Y^, Sf, Zf) + h{p. 



T - T(l-pe) 

As T — cx), Pe — ^ and we can write, 

limsup ^^^Ml < limsup h{W; , Sf, Zf). (14) 



We focus our attention on the right side of Eq. (fT4|) . 



(a) 1 



- V i{w, zt\ Sk, z,\Yt\ st\ zt') - i{zt'; n, Sk, z,\Yt\ st\ zt\ W) 



(J 2 

T 

(c) 1 



'^-^ =0 
T 

5^ Sk, Zk\Yt\ St\ Z\-^) - H{Yk, Sk, Zk\Yt\ St\ Z\-\ W, X^) 

k=l 

T 

k=l 

+ H{Zj,\Yt\ St\ Zt\ Sk, n) - H{Sk\St\ Zt\ W, X^) 

- H{Yk\St\ Zt\ W, Xk, Sk) - H{Zk\St\ Zt\ W, X,, Sk, Y^) 
1 T 

— -f^l^fcl^fc, Xfc) — H{Zk\Xk,Yk) 

' V ' 

(e) 

1 ^ 

= - J2 H{Yk\Yl'~\St\Zt\Sk) - H{Yk\Sk,Uk) + H{Zk\Sk) 

k=l J jj 

Here, (a) and (c) follow from the chain rule of Mutual Information and entropy respectively, 
and (b) is due the fact that the channel input X^ is a function of the channel observations 
and the message W. The above expression can be divided into two parts I and 11. Part I is the 
same as that given by lfT4ll and can be written as a cost that is a function of the coding policy 
p{xk\z'l~^). This cost expression is used to recast the maximization problem (fT4l) as a Markov 
Decision Process. Consequently, the argument for the converse just follows the original proof 
given in [14j. Part II of the expression is a constant that does not depend on the coding policy 
and can be ignored. The direct theorem can be used without modification to complete the proof. 
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Fig. 1. Block diagram showing the system model considered in this work. 
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Fig. 2. Block diagram showing the system model used for the information theoretic capacity calculations for the FSMC with 
CSI at the receiver and feedback. 
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Fig. 3. The Erasure Channel with Vacations (ECV). 'V is a zero-cost, zero-information symbol. {'r,'2' 
unit cost on transmission. The transition labels represent the appropriate probabilities. 
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Fig. 4. The power allocation policy as a function of the power constraint V for Case 1 of the 2SMEC with perfect causal CSI. 
The solid (dashed) trace corresponds to the power allocation for s — 1(2). 
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Channel Capacity (Non-Causal Perfect CSI) 

Channel Capacity (Cauaal Perfect CSI) 

1 Capacity (ACK/NAK) 
DP Schcdidcr (Non-CaiL^al Perfect CSI) 
DP Scheduler (Causal Perfect CSI) 
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Blind Scheme 
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Fig. 5. Comparison of the energy efficiency performance of various schemes with their theoretical bounds for 2-state Markov 
channel. Here ei — 0.2, £2 ~ 0.8, P21 = 0.1 and pi2 = 0.2. 




Fig. 6. Comparison of the energy efficiency performance of various schemes with their theoretical bounds for 3-state Markov 
channel. Here pi2 = 0.025, P13 — 0.075, P21 = 0.075, P23 ~ 0.05, P31 — 0.05 and P32 = 0.05. The loss probabilities are 
ei = 0.2, £2 = 0.85 and £3 = 0.95. 
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Fig. 7. Comparison of the performance of ACK/NAK scliedulers with different table sizes. Here ti = 0.23, t2 = 0.78, P2i = 
0.09 and pu = 0.18. 
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Fig. 8. Comparison of the performance of ACK/NAK schedulers with different time horizons T. Here ei = 0.1, £2 = 
0.9, P21 = 0.1 andpi2 = 0.2. 
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Fig. 9. Comparison of the performance of ACK/NAK schedulers formulated with different terminal costs C. Here ei = 
0.2, £2 = 0.8, P21 = 0.1 and pu = 0.2. 
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Fig. 10. The maximum power penalty Aab as a function of the channel memory ^ for different values of packet loss probabilities 
{£1,62} with 7r(l) = 1/3 and 7r(2) = 2/3. 
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Fig. 11. The power penalty LdB as a function of the channel memory /i for different values of packet loss probabilities {ei, £2} 
with 7r(l) = 1/3 and n{2) — 2/3. The scheduler attempts to transmit 100 packets over 500 time slots for all cases. 



4.5 




0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 

Memory fi 



Fig. 12. The power penalty idB as a function of the channel memory /i for different values of steady state probabilities 
{7r(l), 7r(2)} with ei — 0.1 and £2 ~ 0.9. The scheduler attempts to transmit 50 packets over 500 time slots for all cases. 
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